Robot apparatus, and behavior controlling method for robot apparatus

ABSTRACT

A robot ( 1 ) is provided which includes a situated behaviors layer (SBL) ( 58 ). This SBL ( 58 ) is formed in the form of a tree structure in which a plurality of schemata (behavior modules) is connected hierarchically in such a matter that the schemata are highly independent of each other for each of them to behave uniquely. A patent schema can define a pattern in which child schemata are connected, such as an OR type pattern in which the child schemata are caused to behave uniquely, AND type pattern in which the plurality of child schemata are caused to behave simultaneously or a SEQUENCE type pattern indicating a sequence in which the plurality of child schemata should behave, thereby permitting to select a behavior pattern of the robot ( 1 ). Also, a new child schema can additionally be included in the SBL ( 58 ) without having to rewrite the schemata connection in the tree structure, whereby a new behavior or function can be added to the robot ( 1 ). Namely, the plurality of behavior modules permits to enable the robot ( 1 ) to show a complicated behavior and have units thereof recombined.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to a robot apparatus which canbehave autonomously to have realistic communications with the user and amethod of controlling the behavior of the robot apparatus, and moreparticularly to an autonomous robot apparatus which can recognizesurroundings thereof including images and sounds and behave itself inresponse to such conditions, and a behavior control method for the robotapparatus.

This application claims the priority of the Japanese Patent ApplicationNo. 2002-257097 filed on Sep. 2, 2002, the entirety of which isincorporated by reference herein.

2. Description of the Related Art

A machine or device capable of performing like a human being (animal) byelectrical or magnetic operations is called “robot”. In Japan, therobots had started prevailing in the late 1960s. Many of the robots wereindustrial robots such as manipulators, transport robots and the likeintended for automation, unmanning, etc. of the factory productionlines.

Recently, there have been developed utility robots which support thehuman life as a partner of the human being, that is, support the humanactivities in the residential environment and other daily livingsituations. Different from the industrial robots, the utility robotshave abilities of autonomically learning how to adapt themselves tohuman beings different in personality from each other in various aspectsof the human beings' living environments or to various environments orto various environments. There are being put to practical use the “pet”robots simulating the physical mechanism and behaviors (motions oractions) of a quadrupedal walking animal such as a dog or cat and the“humanoid” robots designed based on the physical mechanism and behaviorsof the human being or the like walking on two feet, for example.

Since the above “pet” and “humanoid” robots can perform variousbehaviors designed with major consideration to the entertainment ascompared with the industrial robots, they are often called“entertainment robots”. Some of the entertainment robots autonomouslybehave adaptively to external information and internal status.

Generally, the robot of such an autonomous type selects a sequence ofbehaviors correspondingly to change in environments including images andsounds. Also, some of the autonomous robots have other behaviorselection mechanisms which models emotions such as instinct and feelingfor managing the internal status of the system and selecting a behaviorcorrespondingly to a change of the internal status. It should be notedthat the system internal status is changed as the environment changesand also when the robot does the selected behavior.

Since a robot has to be designed with considerations given to resourcessuch as a hardware and software and required behavior of the robot, manyof behavior modules are implemented on demand.

Actually, a behavior as a whole can be implemented monolithically, thatis, by one software module. To implement a more complicated behavior,however, a behavior may be modularized, namely, it may be decomposedinto a plurality of modules to implement the behavior by interactionsamong the modules.

However, when any one of the modules is used to implement anotherbehavior, the procedure of interaction between the modules and themodules themselves have to radically be rewritten for that purpose insome cases. The modules are difficult to recombine.

OBJECT AND SUMMARY OF THE INVENTION

It is therefore an object of the present invention to overcome theabove-mentioned drawbacks of the related art by providing a robotapparatus capable of implementing a complicated behavior by a pluralityof behavior modules and in which modules can easily be recombined, and abehavior control method for the robot apparatus.

The above object can be attained by providing a robot apparatuscomprising:

means for recognizing the environment around the robot apparatus;

means for managing an internal status of the robot apparatus accordingto the recognized environment and/or a behavior of the robot apparatus;and

a plurality of performing means for performing behaviors of the robotapparatus according to the environment and/or internal status, each ofthe behaviors being performed respectively, wherein

the plurality of performing means are constructed by tree-structureaccording to levels of the behaviors of the robot apparatus, and

lower-order ones of the performing means in the tree-structure performbehaviors of the robot apparatus based on behavior information which isset by higher-order ones of the performing means in the tree-structurewhere the lower-order ones of the performing means are connected.

The “behavior information” referred to herein includes information abouttargets of the behaviors and what the behaviors are.

Also the above object can be attained by providing a robot apparatuscomprising:

means for recognizing the environment around the robot apparatus;

means for managing an internal status of the robot apparatus accordingto the recognized environment and/or a behavior of the robot apparatus;and

a plurality of performing means for performing behaviors of the robotapparatus according to the environment and/or internal status, each ofthe behaviors being performed respectively, wherein

the plurality of performing means are constructed by tree-structureaccording to levels of the behaviors of the robot apparatus,

higher-order ones of the performing means in the tree-structure are ableto set a connection pattern of lower-order ones of the performing meansin the tree-structure, and

the lower-order ones of the performing means perform behaviors accordingto the connection pattern.

As the “connection pattern”, there is available a one indicating abehavior sequence of the plurality of lower-order ones of the performingmeans, a one indicating that the plurality of lower-order ones of theperforming means are caused to move simultaneously or a one indicatingthat each of the lower-order ones of the performing means performsbehaviors respectively.

In the above robot apparatus, the performing means which respectivelyperform behaviors, are highly independent of each other, and theperforming means perform behaviors based on behavior information whichis set by higher-order ones of the performing means. Also, higher-orderones of the performing means are able to set a connection pattern oflower-order ones of the performing means, and the lower-order ones ofthe performing means perform behaviors according to the connectionpattern.

Also the above object can be attained by providing a behaviorcontrolling method for a robot apparatus, wherein

the method comprising a plurality of performing modules for performingbehaviors of the robot apparatus respectively according to recognizedenvironment around the robot apparatus, and/or an internal status of therobot apparatus according to the recognized environment and/or abehavior of the robot apparatus,

the plurality of performing modules are constructed by tree-structureaccording to levels of the behaviors of the robot apparatus, and

lower-order ones of the performing modules in the tree-structure performbehaviors of the robot apparatus based on behavior information which isset by higher-order ones of the performing modules in the tree-structurewhere the lower-order ones of the performing modules are connected.

The “behavior information” referred to herein includes information abouttargets of the behaviors and what the behaviors are.

Also the above object can be attained by providing a behaviorcontrolling method for a robot apparatus, wherein

the method comprising a plurality of performing modules for performingbehaviors of the robot apparatus respectively according to recognizedenvironment around the robot apparatus, and/or an internal status of therobot apparatus according to the recognized environment and/or abehavior of the robot apparatus,

the plurality of performing modules are constructed by tree-structureaccording to levels of the behaviors of the robot apparatus,

higher-order ones of the performing modules in the tree-structure areable to set a connection pattern of lower-order ones of the performingmodules in the tree-structure, and

the lower-order ones of the performing modules perform behaviorsaccording to the connection pattern.

As the “connection pattern”, there is available a one indicating abehavior sequence of the plurality of lower-order ones of the performingmodules, a one indicating that the plurality of lower-order ones of theperforming modules are caused to move simultaneously or a one indicatingthat each of the lower-order ones of the performing modules performsbehaviors respectively.

In the above behavior controlling method for a robot apparatus, theperforming modules which respectively perform behaviors, are highlyindependent of each other, and the performing modules perform behaviorsbased on behavior information which is set by higher-order ones of theperforming modules. Also, higher-order ones of the performing modulesare able to set a connection pattern of lower-order ones of theperforming modules, and the lower-order ones of the performing modulesperform behaviors according to the connection pattern.

These objects and other objects, features and advantages of the presentinvention will become more apparent from the following detaileddescription of the preferred embodiments of the present invention whentaken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of the robot apparatus according to thepresent invention, showing the appearance and construction of the robotapparatus;

FIG. 2 schematically illustrates a degree-of-freedom construction modelof the robot apparatus shown in FIG. 1;

FIG. 3 is a block diagram of the robot apparatus;

FIG. 4 shows a basic architecture of the behavior control systemincluded in the robot apparatus to control behaviors of the robotapparatus;

FIG. 5 schematically illustrates objects of the behavior control system;

FIG. 6 schematically illustrates a mode of situated behavior control bya situated behaviors layer of the behavior control system;

FIG. 7 schematically illustrates the situated behaviors layer composedof a plurality of schemata;

FIG. 8 schematically illustrates a tree structure of the schemataincluded in the situated behaviors layer;

FIG. 9 shows the tree structure having added thereto focus informationfor each schema;

FIG. 10 shows a sub-tree structure of an Approach Target behavior inwhich Tracking and Approach schemata are focused on different targets,respectively;

FIG. 11 schematically illustrates behaviors of the robot apparatus whenTracking and Approach schemata are focused on different targets,respectively;

FIG. 12 shows a sub-tree structure of the Approach Target behavior inwhich the Approach Target schema sets the same focus for the Trackingand Approach schemata;

FIG. 13 shows a sub-tree structure of a Search Target behavior in whichchild schemata are connected in an OR type pattern;

FIG. 14 shows a tree-structure of the Approach Target behavior in whichchild schemata are connected in an AND type pattern;

FIG. 15 shows a sub-tree structure of the Approach Target behavior inwhich the Approach Target schema sets the same focus for the childschemata;

FIG. 16 schematically illustrates a behavior of the robot apparatus whenthe Approach Target schema sets the same focus for the child schemata;

FIG. 17 shows a tree structure in which the Search Target schema,Approach Target schema and a Dialogue schema are connected in a SEQUENCEtype pattern;

FIG. 18 shows a sub-tree structure of the Approach Target behavior inwhich a Navigation schema is disposed subordinately to the Approachschema; and

FIG. 19 schematically illustrates a behavior of the robot apparatus whenthe Navigation schema is disposed subordinately to the Approach schema.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention will be described in detail belowwith reference to the accompanying drawings.

The embodiment of the present invention is a bipedal walking robot. Thisrobot is a utility robot which can help the human activities in variousliving environments and various phases of daily life, and it is also anentertainment robot capable of behaving adaptively to an internal status(anger, sadness, joy, happiness, etc.) and also performing the basicbehaviors (motions or actions) of a human being.

Referring now to FIG. 1, the bipedal walking robot according to thepresent invention is schematically illustrated in the form of aperspective view. As shown, the robot, generally indicated with areference 1, consists of a body unit 2, head unit 3, two arm units,right and left, 4R and 4L and two leg units, right and left, 5R and 5L.The head unit 3 is coupled in place to the body unit 2, and the armunits 4R and 4L and leg units 5R and 5L are coupled in place to the bodyunit 2. It should be noted that the “R” and “L” stand for “right” and“L”, respectively, and so the arm units 4R and 4L, for example, will bereferred to as “arm units 4R/L” hereunder wherever appropriate.

The robot 1 has degrees of freedom as schematically illustrated in FIG.2. The neck unit 3 has three degrees of freedom including a neck jointyaw axis 101, neck joint pitch axis 102 and a neck joint roll axis 103.

Each of the arm units 4R/L forming the upper limbs has a should jointpitch axis 107, shoulder joint roll axis 108, upper arm yaw axis 109,elbow joint pitch shaft 110, lower arm yaw axis 111, wrist joint pitchaxis 112, wrist joint roll axis 113 and a hand 114. The hand 114 isactually a multi-joint, multi-degrees-of-freedom structure including aplurality of fingers. Since any behavior of the hand 114 is lesscontributed to, or less influences, the control of the posture andwalking of the robot 1, however, the hand 114 is assumed herein to haveno degree of freedom. Therefore, each of the arm units 4R/L has sevendegrees of freedom.

Also, the body unit 2 three degrees of freedom including a body pitchaxis 104, body roll axis 105 and a body yaw axis 106.

Each of the leg units 5R/L forming the lower limbs has a hip joint yawaxis 115, hip joint pitch axis 116, hip joint roll axis 117, knee jointpitch axis 118, ankle joint pitch axis 119, ankle joint roll axis 120and a foot 121. It is assumed herein that the intersection between thehip joint pitch axis 116 and hip joint roll axis 117 defines the hipjoint position of the robot 1. It should be noted that although the foot121 of the human being is actually a structure including a sole havingmultiple joints and multiple degrees of freedom, the foot sole of therobot 1 is assumed herein to have no degree of freedom. Therefore, eachof the leg-units 5R/L has six degrees of freedom.

Namely, the robot 1 has a total of thirty two degrees of freedom(=3+7×2+3+6×2). However, the entertainment robot 1 is not always limitedto the thirty two degrees of freedom. Depending upon the constraints indesigning and manufacture and required specifications, the number ofdegrees of freedom, that is, joints, may of course be increased ordecreased appropriately.

Each of the above degrees of freedom the robot 1 has is actuallyimplemented by an actuator. The actuator should desirably be small andlightweight to meet the requirements that the robot 1 should be formedto have a shape near the human being's natural shape with as lessexcessive bulging-out as possible and the posture of the bipedal walkingrobot as an unstable structure should be well controllable.

FIG. 3 schematically illustrates a control system included in the robot1. As shown, the robot 1 includes the body unit 2, head unit 3 and thearm units 4R/L and leg units 5R/L as the four limbs of a human being,and a control unit 10 which provides an adaptive control forcoordinating behaviors of the bodily units.

The control unit 10 makes centralized control over the behaviors of therobot 1 as a whole. The control unit 10 is composed of a main controller111 including main circuit components such as a CPU (central processingunit), DRAM, flash ROM, etc., and a peripheral circuit 12 including apower circuit; interfaces (not shown) for transfer of data and commandsto and from various components of the robot 1, etc.

The control unit 10 may not be installed in any limited location.Although the control unit 10 is installed in the body unit 2 as shown inFIG. 3, it may be installed in the head unit 3. Alternatively, thecontrol unit 10 may be disposed outside the robot 1 in such a mannerthat it can make cable or wireless communications with the robot 1.

Each of the degrees of freedom in the robot 1, shown in FIG. 2, isimplemented by an actuator. That is, the head unit 3 has a neck jointyaw-axis actuator A₂, neck joint pitch-axis actuator A₃ and a neck jointroll-axis actuator A₄ for the neck joint yaw axis 101, neck joint pitchaxis 102 and neck joint roll axis 103, respectively.

In addition, the head unit 3 is provided with a CCD (charge-coupleddevice) camera to capture external situations, a distance sensor tomeasure the distance to an object existing in front of the robot 1, amicrophone to collect external sounds, a speaker to provide outputsounds, a touch sensor to detect a pressure applied due to an physicalaction such as “patting” or “hitting” by the user, etc.

The body unit 2 has a body pitch-axis actuator A₅, body roll-axisactuator. A₆ and a body yaw-axis actuator A₇ for the body pitch axis104, body roll axis 105 and body yaw axis 106, respectively. Also, thebody unit 2 has installed therein a battery which supplies a power tothe robot 1. The battery is a rechargeable one.

The arm units 4R/L are composed of upper arm units 4 ₁R/L, elbow jointunits 4 ₂R/L and a lower arm units 4 ₃R/L, respectively, and each of theupper arm units 4R/L has a shoulder joint pitch-axis actuator A₈,shoulder joint roll-axis actuator A₉, upper arm yaw-axis actuator A₁₀,elbow joint pitch-axis actuator A₁₁, elbow joint roll-axis actuator A₁₂,wrist joint pitch-axis actuator A₁₃ and a wrist joint roll-axis actuatorA₁₄ for the shoulder joint pitch axis 107, shoulder joint roll axis 108,upper arm yaw axis 109, elbow joint pitch shaft 110, lower arm yaw axis111, wrist joint pitch axis 112 and wrist joint roll axis 113,respectively.

The leg units 5R/L are composed of femoral units 5 ₁R/L, knee units 5₂R/L and tibial units 5 ₃R/L, respectively, and each of the leg units5R/L has a hip joint yaw-axis actuator A₁₆, hip joint roll-axis actuatorA₁₇, hip-joint roll-axis actuator A₁₈, knee joint pitch-axis actuatorA₁₉, ankle joint pitch-axis actuator A₂₀ and an ankle jointroll-actuator A₂₁ for the hip joint yaw axis 115, hip joint pitch axis116, hip joint roll axis 117, knee joint pitch axis 118, ankle jointpitch axis 119 and ankle joint roll axis 120, respectively. Each of theactuators A₂, A₃, . . . used for the joints should preferably beformable from a small AC servo actuator connectable directly to a gear,having a servo control system formed as a single chip, and built in amotor unit.

The component units such as the body unit 2, head unit 3, arm units 4R/Land leg units 5R/L are provided with sub controllers 20, 21, 22R/L and23R/L, respectively, to drive and control the corresponding actuators.Further, the leg units 5R/L are provided with grounding check sensors30R/L, and the body unit 2 has installed therein a posture sensor 31 tomeasure the posture of the robot 1.

The grounding check sensor 30R/L is formed from a proximity sensor ormicroswitch provided on the foot sole. The posture sensor 31 is formedfrom a combination of an acceleration sensor and gyro sensor.

Outputs from the grounding check sensors 30R/L show whether each of theright and left feet of the robot 1 walking or running is currently on oroff the ground. Also, an output from the posture sensor 31 shows aninclination or posture of the body unit of the robot 1.

In response to the outputs from the sensors 30R/L and 31, the maincontroller 11 can correct a controlled target dynamically. Morespecifically, the main controller 11 makes an adaptive control of thesub controllers 20, 21, 22R/L and 23R/L to provide a whole-body behaviorpattern in which the upper limbs, body and lower limbs of the robot 1are coordinately driven.

That is, the main controller 11 in the robot 1 sets a foot behavior,zero moment point (ZMP) orbit, body behavior, upper-limb behavior, waistheight, etc. set for a whole-body behavior, generates commands foroperations of the actuators are generated correspondingly to thesettings and transfer the commands to the sub controllers 20, 21, 22R/Land 23R/L. The sub controllers 20, 21, . . . interpret the commandsreceived from the main controller 11 and provide a drive/control signalto each of the actuators A₂, A₃ . . . . The term “ZMP” refers to a pointon the flow where the moment caused by the floor reaction force duringwalking of the robot 1 is zero, and the term “ZMP orbit” is an orbitalong which ZMP moves during walking of the robot 1, for example. Itshould be noted that the concept of ZMP and application of ZMP to thestability criterion of walking robots are referred to the “LeggedLocomotion Robots” by Miomir Vukobratovic.

As above, in the robot 1, the sub controllers 20, 21, . . . interpretcommands received from the main controller 11, and provide adrive/control signal to each of the actuators A₂, A₃, . . . to controlof the operation of each unit. Thus, the robot 1 can shift to a targetposture stably and walk in a stable posture.

The control unit 10 of the robot 1 controls the posture as above, makescentralized processing of video information from various sensorsincluding the acceleration sensor, touch sensor, grounding check sensor,etc. and CCD camera and audio information etc. from the microphone. Thecontrol unit 10 is connected to the main controller 11 via hubs (notshown) various sensors such as the acceleration sensor, gyro sensor,touch sensor, distance sensor, microphone, speaker, etc., actuators, CCDcamera and the battery correspond to.

The main controller 11 sequentially acquires sensor data supplied fromthe aforementioned sensors, video data and audio data, and stores thesedata into place in a DRAM via an internal interface. Also, the maincontroller 11 is sequentially supplied with data on battery's residualpotential from the battery and stores into place in the DRAM. The sensordata, video data and audio data, battery's residual potential data,stored in the DRAM, are utilized by the main controller 11 incontrolling the behaviors of the robot 1.

In the initial status after the robot 1 is turned on, the maincontroller 11 reads a control program and stores it into the DRAM. Also,the main controller 11 measures, based on the sensor data, video andaudio data, battery's residual potential data sequentially stored in theDRAM from the main controller 11, the internal and external statuses andwhether the user has given an instruction to, or has acted on, the robot1.

Further, the main controller 11 selects a behavior depending upon itsinternal status according to the result of judgement and the controlprogram stored in the DRAM, and causes the robot 1 to make a “gesture”by driving corresponding actuators correspondingly to the selectedbehavior.

As above, the robot 1 can control its behavior adaptively to the resultof recognition of any external stimulus and internal status under thecontrol program. FIG. 4 schematically illustrates the basic architectureof a behavior control system 50 adopted in the robot 1.

The behavior control system 50 illustrated can employ an object-orientedprogramming. In this case, each software is handled as a module unitcalled “object” being an integration of data (property) and a procedure(method) for processing the data. Each object permits to transfer theproperty and take over the method by a message communication and aninter-object communication using a common memory.

To recognize an environment 61, the behavior control system 50 includesa visual recognition block (Video) 51, audio recognition block (Audio)52 and a tactile recognition block 53.

The visual recognition block (Video) 51 makes image recognitions such asfacial recognition and color recognition on the basis of a capturedimage supplied via an image input unit such as a CCD and extractsfeatures of the image. The visual recognition block 51 is composed of aplurality of objects such as “Multi-Color Tracker”, “Face Detector” and“Face Identify” which will further be described later.

The audio recognition block (Audio) 52 recognizes audio data suppliedvia an audio input unit such as a microphone, extracts features of theaudio data, and recognizes a set of words (text). The audio recognitionblock 52 is composed of a plurality of objects such as “Audio Recog” and“Speech Recog” which will further be described later.

The tactile recognition block (Tactile) 53 receives touch sensor fromthe touch sensor built in the head unit or the like to recognize anexternal stimulus such as “being patted” or “being hit”.

The internal-status manager (ISM) 54 has an instinct model and emotionmodel, which manage the internal status such as instinct and emotion ofthe robot 1 correspondingly to external stimuli (ES) recognized by theaforementioned visual recognition block 51, audio recognition block 52and tactile recognition block 53.

The emotion model and instinct model are supplied with the result ofeach recognition and a record of behaviors and manage the values ofemotion and instinct. The behavior model can refer to the values ofemotion and instinct.

The short-term memory 55 is a functional module to hold, for a shortterm, a target and event recognized in the environment by the visual,audio and tactile recognition blocks 51, 52 and 53. It stores an imagesupplied from the CCD camera for a short term of about 15 seconds, forexample.

The long-term memory 56 is used to hold, for a long term, informationacquired through learning such as the name of a thing. The long-termmemory 56 memorizes an internal-status change associatively with anexternal stimulus applied to a behavior module, for example.

The behaviors of the robot 1 include mainly “reflexive behavior”implemented by a reflexive SBL 59, “situated behavior” implemented by asituated behaviors layer (SBL) 58 and “deliberative behavior”implemented by a deliberative layer 57.

The deliberative layer 57 makes a long-term plan of behaviors of therobot 1 on the basis of the stored contents of the short- and long-termmemories 55 and 56.

A deliberative behavior is to be done by inference and scheming forrealization of the inference according to a given situation or user'sinstruction. Since the inference and scheming take a longer time forprocessing and computation than the time of reaction for maintaining theinteraction between the robot 1 and user, so the robot 1 infers andschemes the realization of the inference by making reflexive andsituated behaviors on the real-time basis while making responsivereactions.

The situated behaviors layer (SBL) 58 controls a behavior responsive toa situation of the robot 1 on the basis of the contents stored in theshort- and long-term memories 55 and 56 and internal status managed bythe internal-status manager (ISM) 54.

The situated behaviors layer 58 has a state machine for each ofbehaviors, and categorizes the results of recognition of externalinformation supplied from the sensors according to the previousbehaviors and situations to have the robot 1 show a behavior. Also, thesituated behaviors layer 58 implements a behavior intended to hold theinternal status within a certain range (also called “homeostasisbehavior”). If the internal status exceeds a designated range, thesituated behaviors layer 58 activates the behavior of the robot 1 sothat a behavior to return the internal status to within the range willeasily take place (actually, it selects a behavior with considerationgiven to both the internal status and external stimuli). The situatedbehavior takes a long time in response than the reflexive behavior aswill be described later.

The reflexive SBL (situated behaviors layer) 59 is a functional moduleto have the robot 1 behave reflexively to an external stimulusrecognized by the aforementioned visual, audio and tactile recognitionblocks 51, 52 and 53.

Basically, the reflexive behavior is supplied directly with results ofrecognition of external stimuli supplied from the sensors, andcategorizes the information to select an output behavior directly. Forexample, a behavior to instantly evade a obstacle detected shoulddesirably be implemented as a reflexive behavior.

In the robot 1 according to the present invention, the short-term memory55 consolidates the results of recognition from the visual, audio andtactile recognition blocks 51, 52 and 53 for temporal and spatialconsistency among the perceptions and supplies the perceptions of eachobject in the environment as short-term memories to the situatedbehaviors layer (SBL) 58 and the like. Also, the short-term memory 55supplies the result of recognition of external stimulus informationsupplied from the sensors directly to the reflexive SBL 59.

Generally, if the robot 1 is controlled by only the situated behaviorslayer (SBL) 58 and deliberative SBL 57, selecting a behavior takingvarious conditions in consideration, it will show a slower response toany stimuli slowly. On this account, the behavior control system 50 ofthe robot 1 is constructed according to the present invention for thesituated behaviors layer (SBL) 58, deliberative SBL 57 and the reflexiveSBL 59 which decides to implement a behavior under a single sensorcondition to go through separate processes in order to decideimplementation of a behavior taking various conditions (internal statusand external stimuli) in consideration.

Note that the aforementioned deliberative SBL 57, situated behaviorslayer (SBL) 58 and reflexive SBL 59 can be stated as higher-orderapplication programs independent upon the hardware configuration of therobot 1. The configuration-dependent actions and reactions controller 60controls directly the hardware of the robot 1 for driving theaforementioned actuators A₂, A₃, . . . according to instructions fromthe higher-order applications (behavior modules called “schema”) forchanging environment around the robot 1.

Each of the functional modules in the behavior control system 50 of therobot 1 as shown in FIG. 4 is constructed as an object. Each objecttransfers data (property) and inherit a program (method) having behaviorof a thing stated therein by a message communication and an inter-objectcommunication using a common memory. Each object will be explained willbe described herebelow with reference to FIG. 5 showing an objectconstruction in the behavior control system 50.

The visual recognition block 51 is composed of three objects “FaceDetector”, “Multi-color Tracker” and “Face Identify”. The Face Detectoris an object which detects the face area in an image frame and outputsthe result of recognition to the Face Identify object. The Multi-colorTracker is an object which recognizes a color and outputs the result ofrecognition to the Face Identify and Short-term Memory (an objectincluded in the short-term memory 55) objects. The Face Identify is anobject which identifies a person by searching an on-hand dictionary fora detected face image and outputs ID information about the person alongwith information about the position and size of the face image area tothe Short-term Memory object.

The audio recognition block 52 is composed of two objects “Audio Recog”and “Speech Recog”. The Audio Recog is an object which receives audio(speech) data from the audio input unit such as a microphone, extracts afeature of the audio data and detects a speech section. It outputs afeature amount of the audio data in the speech section and direction ofsound source to the Speech Recog and Short-term Memory objects. TheSpeech Recog is an object which recognizes a speech using the audiofeature supplied from the Audio Recog object and a speech and syntaxdictionaries and outputs a set of recognized words to the Short-termMemory object.

The tactile recognition block 53 includes an object called “TactileSensor” which recognizes an input from the touch sensor and outputs theresult of recognition to the Short-term Memory object and anInternal-status Manager (ISM) object which manages the internal status.

The Short-term Memory (STM) is an object included in the short-termmemory 55. It is a functional module to hold, for a short term, a targetand event recognized in the environment by each object in theaforementioned recognition system (for example, storing an input imagefrom the CCD for a short term of about 15 seconds). It periodicallygives a notification of an external stimulus to a Situated BehaviorsLayer (SBL) as an STM client.

The objects in the behavior control system 50 includes a Long-termMemory (LTM). The Long-term Memory (LTM) is an object included in thelong-term memory 56, and it is used to hold, for a long term,information acquired through learning such as the name of a thing andthe like. The Long-term Memory can associatively store a change of theinternal status in a behavior module due to an external stimulus, forexample.

The Internal-status Manager (ISM) is an object included in theinternal-status manager 54. It manages several emotions includinginstinct and affect in the form of mathematical models. Morespecifically, it manages the internal status such as the instinct andemotion of the robot 1 on the basis of an external stimulus (ES)recognized by each of the objects in the aforementioned recognitionsystem.

The Situated Behaviors Layer (SBL) is an object included in the situatedbehaviors layer (SBL) 58. That is, it is an object as a client of theShort-term Memory (=STM)). It periodically has a notification ofinformation about external stimuli (target and event) from theShort-term Memory, and selects a schema, namely, a behavior module toperform. The SBL will be described in detail later.

A Reflexive SBL is included in the reflexive SBL 59, and implementsreflexive and direct motions/sounds/leds of the bodily units of therobot 1 in response to an external stimulus recognized by each of theobjects included in the aforementioned recognition system. For example,it provides a behavior to instantly evade a obstacle detected.

As above, the Situated Behaviors Layer (SBL) object selects a behavioron the basis of an external stimulus, change of the internal status,etc. On the other hand, the Reflexive. SBL provides a reflexive behaviorin response to an external stimulus. Since these two objects selectbehaviors independently of each other, the robot 1 performs behaviormodules (schemata) selected by the objects because of a conflict betweenhardware resources in the robot 1. On this account, a Resource Managerobject is included in the behavior control system 50. This objectarbitrates the competition between the hardware sources when theSituated Behaviors Layer and Reflexive SBL select behaviors,respectively. It notifies each object implementing a behavior of abodily unit of the robot 1 of the result of arbitration, and thus therobot 1 is driven.

In addition, there are provided “Sound Performer”, “Motion Controller”and “LED Controller”. They are objects which implement actions of bodilyunits of the robot 1. The Sound Performer object outputs a sound orspeech. It synthesizes a sound correspondingly to a text and commandsupplied from the Situated Behaviors Layer (SBL) via the ResourceManager, and outputs the sound from the speaker on the robot 1. TheMotion Controller object activates the actuators A₂, A₃, . . . of therobot 1. When supplied with a command to move the hand or leg from theSituated Behaviors Layer (SBL) via the Resource Manager, it computes anangle of a corresponding joint. The LED Controller object turns on oroff an LED (light-emitting diode). Supplied with a command from theSituated Behaviors Layer (SBL) via the Resource Manager, it turns on oroff the LED.

FIG. 6 schematically illustrates a mode of situated behavior control bythe aforementioned situated behaviors layer (SBL) 58 (including thereflexive SBL 59). A result of environment recognition by therecognition system (visual, audio and tactile recognition blocks 51, 52and 53) is supplied as an external stimulus to the situated behaviorslayer (SBL) 58. Also a change of the internal status, corresponding tothe result of environment recognition by the recognition system (visual,audio and tactile recognition blocks 51, 52 and 53) is supplied to thesituated behaviors layer (SBL) 58. Then, the situated behaviors layer(SBL) 58 can select a due behavior through judging the situation on thebasis of an external stimulus and internal-status change.

The situated behaviors layer (SBL) 58 has a state machine for each ofbehavior modules, and categorizes the results of recognition of externalstimuli supplied from the sensors depending upon the previous behaviorsand situations to have the robot 1 show a behavior. Each of the behaviormodules is stated as a schema having a “monitor” function to judge thesituation on the basis of an external stimulus and internal status andan “action” function to implement a state transition (state machine)incidental to implementation of a behavior. FIG. 7 schematicallyillustrates how the situated behaviors layer (SBL) 58 is composed of aplurality of schemata.

The situated behaviors layer (SBL) 58 (more strictly saying, a portionof the situated behaviors layer 58 that controls ordinary situatedbehaviors) is formed as a tree structure in which the plurality ofschemata is hierarchically linked to each other. It totally judges amore optimum schema on the basis of an external stimulus andinternal-status change to control a behavior. The tree includes behaviormodels obtained through mathematization of ethological situatedbehaviors, for example, and a plurality of sub trees (or branches) suchas sub trees for expressing affect.

Supposing interactions, as behaviors of the robot 1, with the user orhuman being, the tree system of the situated behaviors layer (SBL) 58 isformed herein with reference to human behaviors actually found.

The typical behaviors performed by a person wanting to dialogue withsomeone are assumed to be a sequence of simple independent ones asfollows:

-   -   (a) He or she searches a partner (target);    -   (b) He approaches the partner (target); and    -   (c) He dialogues with the partner (target).

An example tree structure of the situated behaviors layer (SBL) 58,constructed for implementing the above behaviors, will be describedherebelow with reference to FIG. 8.

As shown in FIG. 8, the situated behaviors layer (SBL) 58 is constructedof a “Root” schema located at the top level and at which it is notifiedof an external stimulus from the short-term memory 55, and otherschemata located at levels below the top level. The schemata aredisposed in a sequence of categories from abstract to concrete. Morespecifically, beneath the “Root” schema, there are disposed schemata“Search Target”, “Approach Target” and “Dialogue” at the same level.Thus, a behavior algorithm is modeled. Below the Search target schema,there are disposed schemata “Look To” and “Look For” for concretebehaviors to search a target. Similarly, below the Approach Targetschema, there are disposed schemata “Tracking” and “Approach” forconcrete behaviors to approach the target. Below the Dialogue schema,there are disposed schemata “Tracking” and “Chat” for concrete behaviorsto dialogue with the target. It should be noted that a schema may bedisposed in a different sub tree as the “Tracking” schema is so.

Each of the schemata in the present invention has a function toimplement a behavior uniquely and it is highly independent of the otherschemata. Thus, when designing a schema, it is possible to focus on onlythe feature of a behavior (motion or action) being designed. Also, it isnot necessary to give any consideration to any other requirementspossibly taking place when the schema actually works in the treestructure. Actually, each schema has a focus as action-relatedinformation and can refer to a thing or person in the environment aroundthe robot 1, for example, which will be a specific target when theschema works. There exists minimum necessary information for the schemato perform, and the aforementioned focus is formed from suchinformation. It should be noted that the schemata may have differentfocuses, respectively.

Since each of the schemata is dependent upon information included in itsfocus, it is independent of the tree structure in which itself and otherschema or schemata are disposed. However, each of the schemata mayimplement a behavior in collaboration with any other schema or schemataas will further be described later.

For example, the Tracking schema tracks the position of a target orobserves the path of the target, and cause the robot 1 to turn the face(head) toward the target. In case the target is a person in theenvironment around the robot 1, when the Tracking schema is carried out,the robot 1 continuously swings the head unit 3 to the right and leftcorrespondingly to the relative positional relation between itself andtarget, thereby permitting to keep the head unit 3 in a position italways faces the target rightly. A symbol to identify a targetrecognized by the visual recognition block 51 in the behavior controlsystem 50 can be used as the focus of the Tracking schema. For example,in case the visual recognition block 51 associates ID information witheach recognized thing (or person) or a detected event, the IDinformation can be used as the focus.

FIG. 9 shows the tree structure shown in FIG. 8, having added theretofocus information for each schema. In the example shown in FIG. 9,almost all the schemata need no other focus information than IDinformation (target ID) about a target or a person in the environmentaround the robot 11 who is the partner in dialogue but only the Dialogueand Chat schemata-need focus information showing the topic of thedialogue. Since the Look For schema needs no specific target, it has nofocus. That is, the Look For schema can search any target in theenvironment around the robot 1.

Also, as seen from FIG. 9, there is no restriction by coherence betweeninformation pieces included in the focus. In other words, each target IDcan deal with different targets (thing or person) in the environmentaround the robot 1. If there is no such coherence between theinformation pieces in the focus, a total behavior of the robot 1 will bea simple combination of action implemented by the schemata. For example,in case the low-order (child) Tracking and Approach schemata takedifferent targets T₁ and T₂ as their foci, for example, in the ApproachTarget behavior as shown in FIG. 10, the robot 1 will walk toward thetarget T₂ while tracking the target T₁ as shown in FIG. 11.

Each schema in the present invention has an interface for transfer ofdata and control signal between the schemata, and the schemata may berecombined in the tree structure for implementation of a complicatedbehavior. Also, each schema can have another schema to interact withchild schemata, that is, the rest of the tree structure. Such a schemawill be referred to as a “parent schema” or “hub schema” hereunder.

The parent schema defines a connection pattern in which child schemataare connected or the child schemata to some extent. On the contrary,each of the child schema is only subject to the connection patterndefined by the parent schema, and thus works independently of the parentschema or works under the control of the patent schema. For example, theschemata can exchange focus information between them via theirinterfaces, and also the Approach target schema, for example, can set afocus for the Tracking schema when the Approach Target schema cancontrol the Tracking schema. In this case, the parent schema monitorsthe focus the child schemata are looking to, and controls the behaviorsimplemented by the child schemata, if requested, by setting foci for thechild schemata, for example. FIG. 12 shows an example setting of theTarget T₁ as foci of the Tracking and Approach schemata as the childschemata of the Approach schema as a patent schema.

As above, each schema can be a terminal node, that is, a child schema,and also a hub node, that is, a parent schema. Also, a parent schema candefines a connection pattern in which the child schemata are connected.The connection pattern will be described in detail herebelow.

In the example shown in FIG. 7, a typical pattern in which the childschemata are connected, namely, an instruction pattern, for each subtree. The connection pattern will be described in sequence.

First, the Search Target schema is a parent schema for the Look For andLook To schemata. These three schemata form together a sub tree of“Search Target” behavior. The Look For schema is to make a behavior ofsearching (for something). It visually searches the face of a person,for example. The schema uses the head unit 3 of the robot 1 to scanthrough the environment for searching the face of the person. It shouldbe noted that the schema may of course search any other target such as athing. The Look To schema controls the robot 1 to turn around inresponse to a sound (look in the direction of the sound). For example,when a voice is heard, the schema controls the robot 3 to turn the headunit 3 toward the voice. Thus, the Look For and Look To schemata areindependent of each other, and can implement a behavior independently ofthe other schema.

On the other hand, in the sub tree of Search Target behavior, the SearchTarget schema uses the Look For and Look To schemata as child schematain order to implement a more complicated behavior. The child schematawork independently of each other without being interfered with(controlled) by the parent schema (Search Target). In other words, incase the sub tree of Search Target behaviors is being executed, therobot 1 is caused by the Look For schema to continuously swing the headunit 3 to the right and left until the face of the person as the targetis detected. Then, the robot 1 ceases to swing the head unit 3 so. Atthe same time, the robot 1 is caused by the Look To schema to turn thehead unit 3 toward a voice heard, if any, in the environment around it.

As above, the Search Target schema works not to control the childschemata but as a neutral hub schema. Such a connection pattern iscalled “OR type pattern” herein. Because of this OR type pattern inwhich child schemata are connected, an arbitrary child schema canimplement a behavior at an arbitrary time without being acted on by anyother schema. The sub tree of Search Target behaviors in this case isshown in FIG. 13.

Note that according to the present invention, it is possible toimplement the above-mentioned complicated behavior as compared with asimple behavior by one schema only by defining a pattern in which thechild schemata are connected without recombining the schemata adaptivelyto any specific situation.

The Approach Target schema is a parent schema of the Tracking andApproach schemata. These three schemata form together a sub tree of“Approach Target” behavior. The Tracking schema is to track the positionof a target or observe the moving path of the target, and cause therobot 1 to turn the face (head unit) toward the target. The Approachschema is a schema which causes the robot 1 to walk toward the target.When the target is a person existing in the environment around the robot1 for example, the Approach schema causes the robot 1 to walk toward theperson to within a predetermined distance from the person. Thus, theTracking and Approach schemata are independent of each other, and canimplement a behavior independently of the other schemata.

In the sub tree of Approach Target behavior, however, the ApproachTarget schema uses the Tracking and Approach schemata as child schematato implement a more complicated behavior. Especially, the ApproachTarget schema checks whether the child schemata work consistently witheach other to assure that:

-   -   (a) All the child schemata work in collaboration with each other        and    -   (b) All the child schemata have the same focus, for example, the        same target.        In other words, when the sub tree of Approach Target behaviors        is executed and the target is a person in the environment around        the robot 1, for example, the robot 1 is caused by the Tracking        schema to continuously swing the head unit 3 to the right and        left correspondingly to the relative positional relation between        itself and target and behaviors of the person and itself,        thereby permitting to keep the head unit 3 in a position it        always faces the target rightly. At the same time, the robot 1        is caused by the Approach schema to walk toward the person.

The Approach Target schema controls the child schemata as above. Such aconnection pattern is referred to as “AND type pattern” herein. In thisAND type pattern of connection, all the child schemata work incollaboration with each other. The sub tree of Approach Target behaviorsin this case is shown in FIG. 14.

Note that according to the present invention, it is possible toimplement the above-mentioned complicated behavior as compared with asimple behavior by one schema only by defining a pattern in which thechild schemata are connected without recombining the schemata adaptivelyto any specific situation.

Also, in the sub tree of Approach Target behavior, all the childschemata connected in the AND type pattern have the same focus. FIG. 15shows an example setting of a target T₂ as the focus of the Tracking andApproach schemata being child schemata by the Approach target schemabeing the parent schema of these child schemata. In this case, the robot1 shows a behavior of walking toward the target T₂ while tracking thetarget T₂ as shown in FIG. 16.

The Dialogue schema is the parent schema of the Tracking and Chatschemata. These three schemata form together a sub tree of Dialoguebehaviors. The pattern in which the child schemata in the sub tree ofDialog behaviors are connected is of the AND type and the sub treeitself is similar to that of the Approach Target. So, the Dialoguebehavior sub tree will not be explained any longer.

As above, the illustrated example is a simulation of a person wanting todialogue with some one. The behavior of the robot 1 is implemented bysub trees of Search Target behaviors, Approach Target behaviors andDialogue behaviors. This complicated behavior should include connectionpatterns that can be performed successively. More specifically, when therobot 1 goes to have a dialogue with a person, the person should bewithin a predetermined reach of the robot 1. To be near the person, therobot 1 has to approach the person. The person should be within theenvironment around the robot 1, and the robot 1 has to search (look for)the person. In other words, the behaviors performed by a person wantingto dialogue with someone are stated a sequence of simple independentones as follows:

-   -   (a) He or she searches a partner (target);    -   (b) He approaches the partner (target); and    -   (c) He dialogues with the partner (target).        The above connection pattern is referred to as “SEQUENCE type        pattern” herein. A tree structure of all these behaviors is        shown in FIG. 17. With this SEQUENCE type pattern, the robot 1        can start a behavior at any of different time points        corresponding to the current situation. For example, when a        person with which the robot 1 is going to have a dialogue is        near the robot 1, the Approach Target behaviors can be omitted.

Note that the connection patterns such as the OR, AND and SEQUENCE typepatterns are referred to herein just by way of example and the presentinvention is not limited such connection patterns.

A schema may be a terminal node in a tree structure while being a hubnode in another tree structure. The interface of the schema sendsinformation such as data and control signal from the schema toautomatically accommodate different situations.

Each of the schemata may be designed to detect child schemata, if any,and define a specific control pattern for the child schemata, ordesigned as a “pure” terminal node. In the latter case, when the childschemata are disposed, a default control pattern, for example, OR typepattern, is used.

A sub tree of Approach Target behaviors is taken as an example forexplanation of how an Approach schema is used as a hub schema in thetree structure. As mentioned above, the Approach schema is a one whichcontrols the robot 1 to walk toward a target until the target, forexample, a person in the environment is within a predetermined distancefrom the robot 1. In such a behavior, however, since no consideration isgiven to any obstacle on the path to the target, the robot 1 willpossibly collide with the obstacle, disabling the robot 1 from arrivingat the target.

On this account, a Navigation schema to have the robot 1 evade anobstacle may be included in the behavior control system 50 to improvethe performance of the robot 1. The Navigation schema is to navigate thedirection in which the robot 1 moves. It checks whether there exists anyobstacle in a direction in which the robot 1 moves, and if it hasdetected an obstacle, it indicates an alternative path, for example, apath evading the obstacle. The Navigation schema may be disposed as achild schema of the Approach schema without having to re-design theApproach schema. Just with the Navigation schema connected in a defaultcontrol panel, for example, in the OR type pattern, by the Approachschema, the Approach and Navigation schemata can work in collaborationwith each other. FIG. 18 shows a sub tree of Approach Target behaviorsin this case. As a result, the robot 1 is caused by the Tracking schemato track a specific person while it is caused by the Approach schema towalk toward the person, as shown in FIG. 19. If an obstacle Ob isdetected, the robot 1 is caused by the Navigation schema to evade theobstacle Ob, for example.

As having previously been described, the situated behaviors layer (SBL)58 in the present invention is formed to have a tree structure in whicha plurality of schemata is connected hierarchically to each other sothat the schemata are highly independent of each other to uniquelyperform a behavior. Thus, the tree structure can be recombined easily.With such a recombination, the parent schema can define a connectionpattern for child schemata such as OR, AND or SEQUENCE type pattern forimplementation of a variety of behavior patterns.

Also, since the schemata are highly independent of each other, a newchild schema can be added without having to rewrite the schemata,whereby a new action or function can be added to the robot 1.

In the foregoing, the present invention has been described in detailconcerning certain preferred embodiments thereof as examples withreference to the accompanying drawings. However, it should be understoodby those ordinarily skilled in the art that the present invention is notlimited to the embodiments but can be modified in various manners,constructed alternatively or embodied in various other forms withoutdeparting from the scope and spirit thereof as set forth and defined inthe appended claims.

Although the embodiment of the present invention has been describedconcerning the bipedal walking robot in the foregoing, the presentinvention is applicable to a robot which acts adaptively to theenvironment and internal status. Namely, the present invention is notlimited to any bipedal walking robots and other legged robots.

1. A robot apparatus comprising: means for recognizing the environmentaround the robot apparatus; means for managing an internal status of therobot apparatus according to the recognized environment and/or abehavior of the robot apparatus; a plurality of performing means forperforming behaviors of the robot apparatus according to the environmentand/or internal status, each of the behaviors being performedrespectively, wherein the plurality of performing means are constructedby tree-structure according to levels of the behaviors of the robotapparatus, and lower-order ones of the performing means in thetree-structure perform behaviors of the robot apparatus based onbehavior information which is set by higher-order ones of the performingmeans in the tree-structure where the lower-order ones of the performingmeans are connected; and means for creating a long term plan ofbehaviors based on stored contents of a short term memory and a longterm memory.
 2. The apparatus as set forth in claim 1, wherein thebehavior information includes information about targets of thebehaviors.
 3. The apparatus as set forth in claim 1, wherein thebehavior information is what the behaviors are.
 4. The apparatus as setforth in claim 1, wherein the plurality of performing means can havelowerorder ones of the performing means connected thereto.
 5. A robotapparatus comprising: means for recognizing the environment around therobot apparatus; means for managing an internal status of the robotapparatus according to the recognized environment and/or a behavior ofthe robot apparatus; a plurality of performing means for performingbehaviors of the robot apparatus according to the environment and/orinternal status, each of the behaviors being performed respectively,wherein the plurality of performing means are constructed bytree-structure according to levels of the behaviors of the robotapparatus, higher-order ones of the performing means in thetree-structure are able to set a connection pattern of lower-order onesof the performing means in the tree-structure, and the lower-order onesof the performing means perform behaviors according to, the connectionpattern; and means for creating a long term plan of behaviors based onstored contents of a short term memory and a lone term memory.
 6. Theapparatus as set forth in claim 5, wherein the connection patternindicates a behavior sequence of the plurality of lower-order ones ofthe performing means.
 7. The apparatus as set forth in claim 5, whereinthe connection pattern indicates that the plurality of lower-order onesof the performing means are caused to move simultaneously.
 8. Theapparatus as set forth in claim 5, wherein the connection patternindicates that each of the lower-order ones of the performing meansperforms behaviors respectively.
 9. The apparatus as set forth in claim5, wherein the plurality of performing means can have lower-order onesof the performing means connected thereto.
 10. A behavior controllingmethod for a robot apparatus, the method comprising: performingbehaviors of the robot apparatus according to recognized environmentaround the robot apparatus, and/or an internal status of the robotapparatus according to the recognized environment and/or a behavior ofthe robot apparatus, wherein a plurality of performing modules areconstructed by tree-structure according to levels of the behaviors ofthe robot apparatus, and lower-order ones of the performing modules inthe tree-structure perform behaviors of the robot apparatus based onbehavior information which is set by higher-order ones of the performingmodules in the tree-structure where the lower-order ones of theperforming modules are connected; and creating a long term plan ofbehaviors based on stored contents of a short term memory and a longterm memory.
 11. The method as set forth in claim 10, wherein thebehavior information includes information about targets of thebehaviors.
 12. The method as set forth in claim 10, wherein the behaviorinformation is what the behaviors are.
 13. The method as set forth inclaim 10, wherein the plurality of performing modules can havelower-order ones of the performing modules connected thereto.
 14. Abehavior controlling method for a robot apparatus, the methodcomprising: performing behaviors of the robot apparatus according torecognized environment around the robot apparatus, and/or an internalstatus of the robot apparatus according to the recognized environmentand/or a behavior of the robot apparatus, wherein a plurality ofperforming modules are constructed by tree-structure according to levelsof the behaviors of the robot apparatus, higher-order ones of theperforming modules in the tree-structure are able to set a connectionpattern of lower-order ones of the performing modules in thetree-structure, and the lower-order ones of the performing modulesperform behaviors according to the connection pattern; and creating along term plan of behaviors based on stored contents of a short termmemory and a long term memory.
 15. The method as set forth in claim 14,wherein the connection pattern indicates a behavior sequence of theplurality of lower-order ones of the performing modules.
 16. The methodas set forth in claim 14, wherein the connection pattern indicates thatthe plurality of lower-order ones of the performing modules are causedto move simultaneously.
 17. The method as set forth in claim 14, whereinthe connection pattern indicates that each of the lower-order ones ofthe performing modules performs behaviors respectively.
 18. The methodas set forth in claim 14, wherein the plurality of performing modulescan have lower-order ones of the performing modules connected thereto.